Variational Problems in Quasi–Newton Methods

It has been known since the early 1970s that the Hessian matrices in quasi– Newton methods can be updated by variational means, in several different ways. The usual formulation of these variational problems uses a coordinate system, and the symmetry of the Hessian matrices are enforced as explicit constraints. As a result, the variational problems seem complicated. In this paper, we give a very simple, coordinate–free, and unified treatment of these variational problems by working directly in the vector space of symmetric matrices. Most of our variational problems become simple, least squares problems. All variational problems are convex programming problems, and we consider their duals. A novel feature of our work is that we construct several new variational problems whose optimal solutions coincide with quasi–Newton update matrices. These new variational problems may be useful in suggesting new quasi–Newton methods in the future.